Conventional Machine Learning Models Research Articles

NOx Emission Predictions in Gas Turbines Through Integrated Data-Driven Machine Learning Approaches

Abstract The reduction of NOx emissions is a paramount endeavor in contemporary engineering and energy production, as these emissions are closely linked to adverse environmental and health impacts. The prediction of NOx emission from gas turbines through several integrated data-driven machine learning methods has been evaluated in study. The study compares the performance of ensemble and conventional machine learning models, demonstrating superior accuracy achieved by the ensemble models. Specifically, the Random Forest model achieved an accuracy rate of 91.68%, XGBoost yielded an accuracy of 91.54%, and CATBoost exhibited the highest accuracy at 92.76%. These findings highlight the capability of data-driven machine learning techniques in enhancing NOx emission predictions in gas turbines. The improved prediction by ensembles can be utilized in the development and implementation of more effective control and mitigation strategies in practical applications. Through the application of these advanced machine learning approaches, the gas turbine industry can play a pivotal role in minimizing its environmental impact while optimizing operational efficiency. This study also provides valuable insights into the effectiveness of ensemble machine learning models, advancing our understanding of their capabilities in addressing the critical issue of NOx emissions from gas turbines.

Interpretable Trend Analysis Neural Networks for Longitudinal Data Analysis

Cohort study is one of the most commonly used study methods in medical and public health researches, which result in longitudinal data. Conventional statistical models and machine learning methods are not capable of modeling the evolution trend of the variables in longitudinal data. In this article, we propose a Trend Analysis Neural Networks (TANN), which models the evolution trend of the variables by adaptive feature learning. TANN was tested on dataset of Kaiuan research. The task was to predict occurrence of cardiovascular events within 2 and 5 years, with three repeated medical examinations during 2008 and 2013. For 2-year prediction, The AUC of the TANN is 0.7378, which is a significant improvement than that of conventional methods, while that of TRNS, RNN, DNN, GBDT, RF, and LR are 0.7222, 0.7034, 0.7054, 0.7136, 0.7160, and 0.7024, respectively. For 5-year prediction, TANN also shows improvement. The experimental results show that the proposed TANN achieves better prediction performance on cardiovascular events prediction than conventional models. Furthermore, by analyzing the weights of TANN, we could find out important trends of the indicators, which are ignored by conventional machine learning models. The trend discovery mechanism interprets the model well. TANN is an appropriate balance between high performance and interpretability.

Performance investigation of osmotically assisted reverse osmosis using explainable machine learning models: A comparative study

Osmotically assisted reverse osmosis (OARO) can treat highly concentrated water and achieve higher recovery rates than reverse osmosis (RO). However, existing mathematical models cannot satisfactorily explain OARO systems due to certain limitations. To explore OARO more dynamically, we aimed to analyze OARO performance based on the laboratory-scale data acquired under different operating conditions, by implementing six machine learning (ML) models with the OARO data, evaluating their predictive capabilities, and investigating the effects of input variables on output variables using Shapley Additive Explanation (SHAP) analysis. In predicting the OARO outputs, the ensemble ML models outperformed the conventional ML models. Incorporating FO/RO membrane datasets significantly enhanced the ML model accuracy and mitigated overfitting issues. Regarding the prediction of water flux, the adjusted determination coefficient values increased by 0.478 for Random Forest and 0.569 for Extreme Gradient Boost in the test step. The SHAP analysis consistently ranked the importance of the input variables for OARO outputs. It revealed the presence of potentially influential but previously unrecognized input variables, such as temperature, for the draw concentration. The results of this study are expected to provide a better understanding of OARO.

Research on Annual Runoff Prediction Model Based on Adaptive Particle Swarm Optimization–Long Short-Term Memory with Coupled Variational Mode Decomposition and Spectral Clustering Reconstruction

Accurate medium- and long-term runoff prediction models play crucial guiding roles in regional water resources planning and management. However, due to the significant variation in and limited amount of annual runoff sequence samples, it is difficult for the conventional machine learning models to capture its features, resulting in inadequate prediction accuracy. In response to the difficulties in leveraging the advantages of machine learning models and limited prediction accuracy in annual runoff forecasting, firstly, the variational mode decomposition (VMD) method is adopted to decompose the annual runoff series into multiple intrinsic mode function (IMF) components and residual sequences, and the spectral clustering (SC) algorithm is applied to classify and reconstruct each IMF. Secondly, an annual runoff prediction model based on the adaptive particle swarm optimization–long short-term memory network (APSO-LSTM) model is constructed. Finally, with the basis of the APSO-LSTM model, the decomposed and clustered IMFs are predicted separately, and the predicted results are integrated to obtain the ultimate annual runoff forecast results. By decomposing and clustering the annual runoff series, the non-stationarity and complexity of the series have been reduced effectively, and the endpoint effect of modal decomposition has been effectively suppressed. Ultimately, the expected improvement in the prediction accuracy of the annual runoff series based on machine learning models is achieved. Four hydrological stations along the upper reaches of the Fen River in Shanxi Province, China, are studied utilizing the method proposed in this paper, and the results are compared with those obtained from other methods. The results show that the method proposed in this article is significantly superior to other methods. Compared with the APSO-LSTM model and the APSO-LSTM model based on processed annual runoff sequences by single VMD or Wavelet Packet Decomposition (WPD), the method proposed in this paper reduces the RMSE by 40.95–80.28%, 25.26–57.04%, and 15.49–40.14%, and the MAE by 24.46–80.53%, 16.50–59.30%, and 16.58–41.80%, in annual runoff prediction, respectively. The research has important reference significance for annual runoff prediction and hydrological prediction in areas with data scarcity.

Artificial intelligence combined with high-throughput calculations to improve the corrosion resistance of AlMgZn alloy

Efficiently designing lightweight alloys with combined high corrosion resistance and mechanical properties remains an enduring topic in materials engineering. Due to the inadequate accuracy of conventional stress-strain machine learning (ML) models caused by corrosion factors, a novel reinforcement self-learning ML algorithm combined with calculated features (accuracy R2 >0.92) is developed. Based on the ML models, calculated work functions and mechanical moduli, a Computation Designed Corrosion-Resistant Al alloy is fabricated and verified. The performance (elongation reaches ∼30%) is attributed to the H trapping Al-Sc-Cu phases (-1.44 eV H−1) and Cu-modified η/η' precipitates inside the grain boundaries (GBs).

Exploration and demonstration of explainable machine learning models in prosthetic rehabilitation-based gait analysis.

Quantitative gait analysis is important for understanding the non-typical walking patterns associated with mobility impairments. Conventional linear statistical methods and machine learning (ML) models are commonly used to assess gait performance and related changes in the gait parameters. Nonetheless, explainable machine learning provides an alternative technique for distinguishing the significant and influential gait changes stemming from a given intervention. The goal of this work was to demonstrate the use of explainable ML models in gait analysis for prosthetic rehabilitation in both population- and sample-based interpretability analyses. Models were developed to classify amputee gait with two types of prosthetic knee joints. Sagittal plane gait patterns of 21 individuals with unilateral transfemoral amputations were video-recorded and 19 spatiotemporal and kinematic gait parameters were extracted and included in the models. Four ML models-logistic regression, support vector machine, random forest, and LightGBM-were assessed and tested for accuracy and precision. The Shapley Additive exPlanations (SHAP) framework was applied to examine global and local interpretability. Random Forest yielded the highest classification accuracy (98.3%). The SHAP framework quantified the level of influence of each gait parameter in the models where knee flexion-related parameters were found the most influential factors in yielding the outcomes of the models. The sample-based explainable ML provided additional insights over the population-based analyses, including an understanding of the effect of the knee type on the walking style of a specific sample, and whether or not it agreed with global interpretations. It was concluded that explainable ML models can be powerful tools for the assessment of gait-related clinical interventions, revealing important parameters that may be overlooked using conventional statistical methods.

Comparative analysis of conventional and ensemble machine learning models for predicting split tensile strength in thermal stressed SCM-blended lightweight concrete

This study aimedto predict the split tensile strength (STS) of lightweight concrete having silica fume (SF) as supplementary cementitious material (SCM) under thermal stresses. Machine learning (ML) models including a conventional machine learning (CML) model as Artificial Neural Networks (ANN) and ensemble machine learning (EML) models, particularly Random Forest (RF) and Gradient Boosting (GB) were developed to predict the STS. Dataset comprising of 96 instances was splitted into 80% for training and 20% for testing phase respectively. With the help of the 80% training data and subsequent hyperparameter tuning, various CML and EML models were developed. ML models were analysed statistically for both the training and testing phases, under the model performanceparameters, line diagrams, and Taylor diagrams. After the thorough statistical analysis, it was concluded that the GB as an EML model outperformed all other ML models. This was demonstrated by the fact that it had constantly higher model performance parameter values of RMSE, MAE and R2 as 0.151, 0.115 and 0.977 for training and 0.23, 0.196 and 0.908 for testing respectively. Also, from the line diagrams which shows a good correlation with the actual and predicted STS values and from the Taylor diagram of the various ML models for both the phases, it was concluded that the GB model is found superior model for this specific study.

Estimation of aquatic ecosystem health using deep neural network with nonlinear data mapping

Estimation of aquatic ecosystem health indices can assist in reducing the burden of time-consuming, labor-intensive, and cost-effective fieldwork for the sustainable evaluation of freshwater ecosystem status. In this study, we developed a deep neural network to estimate the trophic diatom index (TDI), benthic macroinvertebrate index (BMI), and fish assessment index (FAI) using water quality and hydraulic and hydrological data. A convolutional neural network (CNN) model was built to estimate health indices. In addition, an autoencoder was adopted to produce manifold features that were used as inputs for the CNN model. Conventional machine learning models, including artificial neural networks, support vector machines, random forests, and extreme gradient boosting, have been developed to estimate the TDI, BMI, and FAI. The results showed that the CNN with an autoencoder exhibited the best performance, with validation accuracies of Nash Sutcliffe Efficiency (NSE) and root mean squared error (RMSE) values of 0.592 and 17.249 for TDI, 0.669 and 12.282 for BMI, and 0.638 and 13.897 for FAI, respectively. The autoencoder enhanced the nonlinear feature learning of the time series and static input data, which contributed to improving the CNN feature extraction for accurate estimation of aquatic ecosystem health indices compared to other data-driven approaches. Therefore, deep learning techniques can be used to investigate aquatic ecosystem health by successfully reflecting the quantitative and qualitative features of health indices.

Sentiment analysis with hotel customer reviews using FNet

Recent research has focused on opinion mining from public sentiments using natural language processing (NLP) and machine learning (ML) techniques. Transformer-based models, such as bidirectional encoder representations from transformers (BERT), excel in extracting semantic information but are resourceintensive. Google’s new research, mixing tokens with fourier transform, also known as FNet, replaced BERT’s attention mechanism with a non-parameterized fourier transform, aiming to reduce training time without compromising performance. This study fine-tuned the FNet model with a publicly available Kaggle hotel review dataset and investigated the performance of this dataset in both FNet and BERT architectures along with conventional machine learning models such as long short-term memory (LSTM) and support vector machine (SVM). Results revealed that FNet significantly reduces the training time by almost 20% and memory utilization by nearly 60% compared to BERT. The highest test accuracy observed in this experiment by FNet was 80.27% which is nearly 97.85% of BERT’s performance with identical parameters.

Comparative study of machine learning models for predicting the compressive strength of concrete using Non-Destructive Testing methods

This study proposes and compares the conventional machine learning (CML) models as artificial neural networks (ANN), support vector machines (SVM) and ensemble machine learning (EML) models namely, random forest (RF), and gradient boosting (GB). These four-machine learning (ML) models were used to predict the compressive strength (CS) of concrete based on two non-destructive testing (NDT) methods, namely ultrasonic pulse velocity (UPV) and rebound hammer number (RHN). The article uses a large and reliable database of experimental data, collected from the literature to train and test the models, and evaluates their performance using various metrics, scatter plots and a Taylor diagram. It was found that gradient boosting (GB) is the most accurate and robust model for this study, followed by RF model, while ANN and SVM have lower performance. For this study, it suggests that EML models are more suitable and reliable than CML models for predicting the CS of concrete based on NDT methods.

Evaluation of machine learning models for cytochrome P450 3A4, 2D6, and 2C9 inhibition.

Cytochrome P450 (CYP) enzymes are involved in the metabolism of approximately 75% of marketed drugs. Inhibition of the major drug-metabolizing P450s could alter drug metabolism and lead to undesirable drug-drug interactions. Therefore, it is of great significance to explore the inhibition of P450s in drug discovery. Currently, machine learning including deep learning algorithms has been widely used for constructing in silico models for the prediction of P450 inhibition. These models exhibited varying predictive performance depending on the use of machine learning algorithms and molecular representations. This leads to the difficulty in the selection of appropriate models for practical use. In this study, we systematically evaluated the conventional machine learning and deep learning models for three major P450 enzymes, CYP3A4, CYP2D6, and CYP2C9 from several perspectives, such as algorithms, molecular representation, and data partitioning strategies. Our results showed that the XGBoost and CatBoost algorithms coupled with the combined fingerprint/physicochemical descriptor features exhibited the best performance with Area Under Curve (AUC) of 0.92, while the deep learning models were generally inferior to the conventional machine learning models (average AUC reached 0.89) on the same test sets. We also found that data volume and sampling strategy had a minor effect on model performance. We anticipate that these results are helpful for the selection of molecular representations and machine learning/deep learning algorithms in the P450 model construction and the future model development of P450 inhibition.

Transcriptomic and neuroimaging data integration enhances machine learning classification of schizophrenia.

Schizophrenia is a polygenic disorder associated with changes in brain structure and function. Integrating macroscale brain features with microscale genetic data may provide a more complete overview of the disease etiology and may serve as potential diagnostic markers for schizophrenia. We aim to systematically evaluate the impact of multi-scale neuroimaging and transcriptomic data fusion in schizophrenia classification models. We collected brain imaging data and blood RNA sequencing data from 43 patients with schizophrenia and 60 age- and gender-matched healthy controls, and we extracted multi-omics features of macroscale brain morphology, brain structural and functional connectivity, and gene transcription of schizophrenia risk genes. Multi-scale data fusion was performed using a machine learning integration framework, together with several conventional machine learning methods and neural networks for patient classification. We found that multi-omics data fusion in conventional machine learning models achieved the highest accuracy (AUC ~0.76-0.92) in contrast to the single-modality models, with AUC improvements of 8.88 to 22.64%. Similar findings were observed for the neural network, showing an increase of 16.57% for the multimodal classification model (accuracy71.43%) compared to the single-modal average. In addition, we identified several brain regions in the left posterior cingulate and right frontal pole that made a major contribution to disease classification. We provide empirical evidence for the increased accuracy achieved by imaging genetic data integration in schizophrenia classification. Multi-scale data fusion holds promise for enhancing diagnostic precision, facilitating early detection and personalizing treatment regimens in schizophrenia.

Leveraging distant supervision and deep learning for twitter sentiment and emotion classification

AbstractNowadays, various applications across industries, healthcare, and security have begun adopting automatic sentiment analysis and emotion detection in short texts, such as posts from social media. Twitter stands out as one of the most popular online social media platforms due to its easy, unique, and advanced accessibility using the API. On the other hand, supervised learning is the most widely used paradigm for tasks involving sentiment polarity and fine-grained emotion detection in short and informal texts, such as Twitter posts. However, supervised learning models are data-hungry and heavily reliant on abundant labeled data, which remains a challenge. This study aims to address this challenge by creating a large-scale real-world dataset of 17.5 million tweets. A distant supervision approach relying on emojis available in tweets is applied to label tweets corresponding to Ekman’s six basic emotions. Additionally, we conducted a series of experiments using various conventional machine learning models and deep learning, including transformer-based models, on our dataset to establish baseline results. The experimental results and an extensive ablation analysis on the dataset showed that BiLSTM with FastText and an attention mechanism outperforms other models in both classification tasks, achieving an F1-score of 70.92% for sentiment classification and 54.85% for emotion detection.

A Review of Machine Learning Techniques in Agroclimatic Studies

The interplay of machine learning (ML) and deep learning (DL) within the agroclimatic domain is pivotal for addressing the multifaceted challenges posed by climate change on agriculture. This paper embarks on a systematic review to dissect the current utilization of ML and DL in agricultural research, with a pronounced emphasis on agroclimatic impacts and adaptation strategies. Our investigation reveals a dominant reliance on conventional ML models and uncovers a critical gap in the documentation of methodologies. This constrains the replicability, scalability, and adaptability of these technologies in agroclimatic research. In response to these challenges, we advocate for a strategic pivot toward Automated Machine Learning (AutoML) frameworks. AutoML not only simplifies and standardizes the model development process but also democratizes ML expertise, thereby catalyzing the advancement in agroclimatic research. The incorporation of AutoML stands to significantly enhance research scalability, adaptability, and overall performance, ushering in a new era of innovation in agricultural practices tailored to mitigate and adapt to climate change. This paper underscores the untapped potential of AutoML in revolutionizing agroclimatic research, propelling forward the development of sustainable and efficient agricultural solutions that are responsive to the evolving climate dynamics.

Detection of Questions from Text Data Using LSTM-Deep Learning Model

This paper discusses the importance of detecting questions in textual data for various applications in natural language processing (NLP), such as question answering and chatbot creation. The proposed approach employs long short-term memory (LSTM) models to accurately identify questions by leveraging the sequential nature of language.The paper highlights that LSTM models address challenges like ambiguous language and varying sentence structures. They allow the model to learn from sequential patterns, crucial for understanding the intent behind the text. The preprocessing steps, including tokenization, embedding, and padding, are detailed to prepare the data for training and testing. The study investigates the impact of hyperparameters like hidden layers, hidden states, and optimizer choice on the LSTM algorithm’s performance. In experiments on benchmark datasets, the proposed LSTM-based approach consistently outperforms conventional machine learning models, achieving a remarkable accuracy of 99.25% on the test dataset. The paper concludes by suggesting future directions, including applyingthe approach to other NLP tasks like named entity recognition, sentiment analysis, and text classification. Further optimization for specific datasets or domains is also encouraged. Overall, this research contributes to robust question detection models in NLP, with potential applications in various fields.  

Biomimetic Deep Learning Networks With Applications to Epileptic Spasms and Seizure Prediction.

In this study, we present a novel biomimetic deep learning network for epileptic spasms and seizure prediction and compare its performance with state-of-the-art conventional machine learning models. Our proposed model incorporates modular Volterra kernel convolutional networks and bidirectional recurrent networks in combination with the phase amplitude cross-frequency coupling features derived from scalp EEG. They are applied to the standard CHB-MIT dataset containing focal epilepsy episodes as well as two other datasets from the Montefiore Medical Center and the University of California Los Angeles that provide data of patients experiencing infantile spasm (IS) syndrome. Overall, in this study, the networks can produce accurate predictions (100%) and significant detection latencies (10 min). Furthermore, the biomimetic network outperforms conventional ones by producing no false positives. Biomimetic neural networks utilize extensive knowledge about processing and learning in the electrical networks of the brain. Predicting seizures in adults can improve their quality of life. Epileptic spasms in infants are part of a particular seizure type that needs identifying when suspicious behaviors are noticed in babies. Predicting epileptic spasms within a given time frame (the prediction horizon) suggests their existence and allows an epileptologist to flag an EEG trace for future review.

Microstructure-Sensitive Deformation Modeling and Materials Design with Physics-Informed Neural Networks

Microstructure-sensitive materials design has become popular among materials engineering researchers in the last decade because it allows the control of material performance through the design of microstructures. In this study, the microstructure is defined by an orientation distribution function. A physics-informed machine learning approach is integrated into microstructure design to improve the accuracy, computational efficiency, and explainability of microstructure-sensitive design. When data generation is costly and numerical models need to follow certain physical laws, machine learning models that are domain-aware perform more efficiently than conventional machine learning models. Therefore, a new paradigm called the physics-informed neural network (PINN) is introduced in the literature. This study applies the PINN to microstructure-sensitive modeling and inverse design to explore the material behavior under deformation processing. In particular, we demonstrate the application of PINN to small-data problems driven by a crystal plasticity model that needs to satisfy the physics-based design constraints of the microstructural orientation space. For the first problem, we predict the microstructural texture evolution of copper during a tensile deformation process as a function of initial texturing and strain rate. The second problem aims to calibrate the crystal plasticity parameters of the Ti-7Al alloy by solving an inverse design problem to match the PINN-predicted final texture prediction and the experimental data.

A novel ensemble model for detecting fake news

<div>Due the growing proliferation of fake news over the past couple of years, our</div><div>objective in this paper is to propose an ensemble model for the automatic classification of article news as being either real or fake. For this purpose, we opt</div><div>for a blending technique that combines three models, namely bidirectional long</div><div>short-term memory (Bi-LSTM), stochastic gradient descent classifier and ridge</div><div>classifier. The implementation of the proposed model (i.e. BI-LSR) on real</div><div>world datasets, has shown outstanding results. In fact, it achieved an accuracy</div><div>score of 99.16%. Accordingly, this ensemble learning has proven to do perform</div><div>better than individual conventional machine learning and deep learning models</div><div>as well as many ensemble learning approaches cited in the literature.</div>

Speaker identification under noisy conditions using hybrid convolutional neural network and gated recurrent unit

<p><span>Speaker identification is biometrics that classifies or identifies a person from other speakers based on speech characteristics. Recently, deep learning models outperformed conventional machine learning models in speaker identification. Spectrograms of the speech have been used as input in deep learning-based speaker identification using clean speech. However, the performance of speaker identification systems gets degraded under noisy conditions. Cochleograms have shown better results than spectrograms in deep learning-based speaker recognition under noisy and mismatched conditions. Moreover, hybrid convolutional neural network (CNN) and recurrent neural network (RNN) variants have shown better performance than CNN or RNN variants in recent studies. However, there is no attempt conducted to use a hybrid CNN and enhanced RNN variants in speaker identification using cochleogram input to enhance the performance under noisy and mismatched conditions. In this study, a speaker identification using hybrid CNN and the gated recurrent unit (GRU) is proposed for noisy conditions using cochleogram input. VoxCeleb1 audio dataset with real-world noises, white Gaussian noises (WGN) and without additive noises were employed for experiments. The experiment results and the comparison with existing works show that the proposed model performs better than other models in this study and existing works</span><span>.</span></p>

A Hierarchical Deep Learning Architecture for Robust Intrusion Detection in Time-Evolving Security Landscapes

Abstract: This paper introduces a novel architecture for Intrusion Detection Systems (IDS), designed to enhance resilience against adversarial attacks by integrating conventional machine learning (ML) models with Deep Learning (DL) models. The proposed system, termed DLL-IDS, comprises three key components: a DL-based IDS, an adversarial example (AE) detector based on local intrinsic dimensionality (LID), and an ML-based IDS. Initially, a novel AE detector is developed using LID to identify potential adversarial examples. This detector serves as a crucial component in identifying suspicious inputs that may be crafted to deceive the IDS. Subsequently, the system leverages the low transferability of attacks between DL and ML models to establish a robust ML model capable of discerning the maliciousness of potential AEs. The DLL-IDS architecture operates as follows: if incoming traffic is flagged as an AE by the detector, the MLbased IDS is employed to evaluate its maliciousness; otherwise, the DL-based IDS handles the prediction. By integrating both DL and ML approaches, the system benefits from the high prediction accuracy of DL models while exploiting the low susceptibility of ML models to adversarial attacks. Experimental results demonstrate significant enhancements in IDS prediction performance under adversarial conditions, achieving high accuracy with minimal resource consumption. This fusion mechanism effectively combines the strengths of DL and ML models, thereby bolstering the overall robustness of the IDS against sophisticated intrusion attempts.